Facial identification techniques

ABSTRACT

As the use of facial biometrics expands in the commercial and government sectors, the need to ensure that human facial examiners use proper procedures to compare facial imagery will grow. Human examiners have examined fingerprint images for many years such that fingerprint examination processes and techniques have reached a point of general acceptance for both commercial and governmental use. The growing deployment and acceptance of facial recognition can be enhanced and solidified if new methods can be used to assist in ensuring and recording that proper examination processes were performed during the human examination of facial imagery.

RELATED APPLICATION DATA

This application is a Continuation of U.S. patent application Ser. No.15/092,328, filed Apr. 6, 2016, now U.S. Pat. No. 9,881,205, whichclaims the benefit of and priority under 35 U.S.C. § 119(e) to U.S.Patent Application No. 62/168,351, filed May 29, 2015, entitled “FacialIdentification Techniques,” each of which are incorporated herein byreference in their entirety.

BACKGROUND

While both finger and facial images exist as images for processing andcomparison, there are key differences between the two:

-   -   Fingerprint images are black and white while facial imagery can        be color or black and white.    -   Fingerprints are caused by touching a surface (which stabilizes        the capture) while facial images are captured with little or no        control over the subject.    -   Finger details (ridge flows and minutia patterns) are a discrete        type of image pattern whereas facial features vary widely across        gender, race, age, hair, pose, lighting, facial obstructions,        etc.    -   There are many accepted courses and career tracks for the        training of fingerprint examiners, whereas there are no current        public courses for the training of facial examiners.

Human examiners can be presented with facial images for comparisons intwo distinct ways:

-   -   Computer based algorithms can be used to produce a list of        candidates for review after searching a probe image. This is        referred to as “Facial Recognition” or FR.    -   Two images can be manually selected for review.

In both cases, it is up to the human facial examiner to analyze the twoimages and produce a final decision whether the person in both images isof the same person. This process has several commonly used definitions:

-   -   Facial Examination (FE): A formal systematic examination of two        facial images to determine if the same person is shown in both        images. An example of this systematic examination technique,        broadly accepted for use in fingerprint examinations, is called        ACE-V (Analysis, Comparison, Evaluation, and Verification).    -   Facial Identification (FI): The human-based examination of the        similarities and differences between two facial images with the        goal of determining if the same person is shown in both.

In any current operational environment using facial biometrics, the actof the FI is where a human facial examiner (or a group of examinersworking together) is responsible for making a decision as to whether thesame person is shown in two distinct images being compared.

BRIEF DESCRIPTION OF THE DRAWINGS

The exemplary embodiments of the technology will be described in detail,with reference to the following figures wherein:

FIG. 1 shows an exemplary setting of probe image eye locations;

FIG. 2 shows setting candidate image eye locations;

FIG. 3 shows an exemplary Side-by-Side probe and candidate images;

FIG. 4 shows an exemplary comparison of skin;

FIG. 5 shows an exemplary comparison of face/head outlines;

FIG. 6 shows an exemplary comparison of face/head composition;

FIG. 7 shows an exemplary comparison of hair/baldness patterns;

FIG. 8 shows an exemplary comparison of a forehead region;

FIG. 9 shows an exemplary comparison of eyebrows;

FIG. 10 shows an exemplary comparison of eyes;

FIG. 11 shows an exemplary comparison of cheeks;

FIG. 12 shows an exemplary comparison of noses;

FIG. 13 shows an exemplary comparison of ears;

FIG. 14 shows an exemplary comparison of mouths;

FIG. 15 shows an exemplary comparison of chin/jaw lines;

FIG. 16 shows an exemplary comparison of necks;

FIG. 17 shows an exemplary comparison of facial hair;

FIG. 18 shows an exemplary comparison of facial lines;

FIG. 19 shows an exemplary comparison of scars;

FIG. 20 shows an exemplary comparison of facial marks;

FIG. 21 shows an exemplary comparison of alterations;

FIG. 22 shows an exemplary embodiment of embedded synchronizedmagnifiers;

FIG. 23 shows an exemplary embodiment of a increased zoom feature ofembedded synchronized magnifiers;

FIG. 24 shows an exemplary embodiment of an increased zoom of embeddedsynchronized magnifiers.

FIG. 25 shows an exemplary facial identification system;

FIG. 26 is a flowchart showing an exemplary method of performing facialrecognition;

FIG. 27 shows an exemplary report; and

FIGS. 28-37 show examples of accepted anatomical definitions of facialcomponents.

DETAILED DESCRIPTION

Embodiments of systems and methods described herein can operationallyensure that a facial examiner follows specific predefined guidelines orbest practices during a facial identification process. While accreditedand accepted training for facial examiners evolve, an operational ownercan integrate the appropriate guidelines or best practices directly intothe actual facial identification processes used in real operationalenvironments or training courses. Such guidelines or best practices caninclude a list of specific facial components that should be reviewed forevery instance of a human facial examiner performing FI.

In one exemplary embodiment, the list of facial components includes, butis not limited to, one or more of skin, face/head outline, face/headcomposition, hairline or baldness pattern, forehead, eyebrows, eyes,cheeks, nose, ears, mouth, chin or jaw line, neck, facial hair, faciallines, scars, facial marks, and alterations. Other embodiments of thelist can include a different set of fewer (or more) facial components.Such guidelines or best practices can also include instructions for eachof these facial components to train, assist, or guide the facialexaminer when performing the FI for that facial component.

Embodiments of systems and methods described herein directly integratethe list of predefined facial components into the FI process:

-   -   1. Providing a direct on-screen reference to the list of facial        components and their associated anatomical terminology during        the facial review.    -   2. Displaying the areas on the facial components on the face as        defined by the guidelines or best practices.    -   3. Optionally ensuring the facial examiner looks at every facial        component in the list.    -   4. Audit that every facial component in the list is reviewed by        the facial examiner.

Exemplary Steps:

Step 1:

The human facial examiner at a workstation sets the eye locations forboth facial images (i.e., probe and candidate images). This operationcan be done by manually reviewing and possibly (re)centering the eyelocations over the left and right eye locations, as shown in FIG. 1 andFIG. 2 (Image 1 104 and Image 2 204). For example, one or more of theeye locations can be dragged-and-dropped to an updated center locationusing an input device such as a mouse. This setting of the eye locationsis done to create landmarks so that all the facial components to bereviewed can be correlated to the eyes.

Step 2:

After the eye locations for both facial images are reviewed and acceptedby the user at the workstation, both images 104 and 204 can be displayedside-by-side, as shown in the exemplary display in FIG. 3.

At this point, two windows 404 and 408 can appear as shown in FIG. 4that are used during the enforced facial identification process.

-   -   1. In a first window 404 (which can optionally be a standalone        window), any referenced document that the operational owner has        decided to utilize in this facial identification process can be        displayed to the user (i.e., facial examiner). The document can        be placed in at any optimum location within the user's computer        desktop environment.    -   2. The predefined list of the facial components mandated by the        guidelines or best practice is displayed to the user in this        window 404, which can also be placed in an optimum location        within the user's computer desktop environment. In addition to        displaying the list of facial components to be examined, other        information, such as an identification information (ID), viewed        status, and the results of the review can be associated with        each facial component and optionally shown in one or more        windows that can optionally be updated as the steps taken by the        examiner are completed. Table 1 below provides an example of        what may be presented to the user in this second standalone        window.        -   IDs may be associated with the facial components that must            be reviewed; the IDs can enumerate and prioritize the facial            components.        -   The default state of Viewed Status can be NO.        -   The default state of the Results (Similarity/Difference) can            be NULL or empty.

TABLE 1 List of Facial Components Similarity (S)/ ID Facial ComponentsViewed Difference (D) A Skin Y/N S/D/NA B Face/Head Outline Y/N S/D/NA CFace/Head Composition Y/N S/D/NA D Hairline/Baldness Pattern Y/N S/D/NAE Forehead Y/N S/D/NA F Eyebrows Y/N S/D/NA G Eyes Y/N S/D/NA H CheeksY/N S/D/NA I Nose Y/N S/D/NA J Ears Y/N S/D/NA K Mouth Y/N S/D/NA LChin/Jaw line Y/N S/D/NA M Neck Y/N S/D/NA N Facial Hair Y/N S/D/NA OFacial Lines Y/N S/D/NA P Scars Y/N S/D/NA Q Facial Marks Y/N S/D/NA RAlterations Y/N S/D/NA

Step 3:

At this point the user can sequence from facial component A through thelast facial component to be examined during the FI process as shown inFIGS. 5-21. In one embodiment, the list of facial components identifiesevery facial component that must be examined for the FI process to beconsidered properly performed and in conformance with a particularstandard, regulation and/or guideline. As the facial components aresequenced:

-   -   Visual boxes (or other highlighting technique(s)) can be shown        to the user overlaid onto the facial images, which represent the        specific facial component to be reviewed (See FIGS. 5-21);    -   The Viewed column is set from “No” to “Yes”, as the user        examines the facial component—this can optionally be        automatically performed as the user/examiner views each specific        facial component and steps to review the next; and    -   The Similarity/Difference column can be set by the user        indicating whether the specific facial component:        -   shows similarity (S),        -   does not show a similarity (D), or        -   cannot be evaluated (N/A).    -   The referenced guideline or best practice document can        optionally be automatically repositioned to show each specific        facial component within the context of the document.

It is to be understood that the order in which the facial components areexamined may differ from that described below. As discussed, FIGS. 5-22shown exemplary comparisons for various facial features. As illustratedin these exemplary Figures, a visual box(s) and/or highlight is placedaround the feature(s) to be compared. Specifically, FIG. 5 shows anexemplary comparison of face/head outlines, with guidelines for thecomparison shown in window 404. FIG. 6 shows an exemplary comparison offace/head composition, with guidelines for the comparison shown inwindow 404. FIG. 7 shows an exemplary comparison of hair/baldnesspatterns, with guidelines for the comparison shown in window 404. FIG. 8shows an exemplary comparison of a forehead region, with guidelines forthe comparison shown in window 404. FIG. 9 shows an exemplary comparisonof eyebrows, with guidelines for the comparison shown in window 404.FIG. 10 shows an exemplary comparison of eyes, with guidelines for thecomparison shown in window 404. FIG. 11 shows an exemplary comparison ofcheeks, with guidelines for the comparison shown in window 404. FIG. 12shows an exemplary comparison of noses, with guidelines for thecomparison shown in window 404. FIG. 13 shows an exemplary comparison ofears, with guidelines for the comparison shown in window 404. FIG. 14shows an exemplary comparison of mouths, with guidelines for thecomparison shown in window 404. FIG. 15 shows an exemplary comparison ofchin/jaw lines, with guidelines for the comparison shown in window 404.FIG. 16 shows an exemplary comparison of neck, with guidelines for thecomparison shown in window 404. FIG. 17 shows an exemplary comparison offacial hair, with guidelines for the comparison shown in window 404.FIG. 18 shows an exemplary comparison of facial lines, with guidelinesfor the comparison shown in window 404. FIG. 19 shows an exemplarycomparison of scars, with guidelines for the comparison shown in window404. FIG. 20 shows an exemplary comparison of facial marks, withguidelines for the comparison shown in window 404. FIG. 21 shows anexemplary comparison of alterations, with guidelines for the comparisonshown in window 404.

Step 4:

FIG. 22 shows an exemplary embodiment of embedded synchronizedmagnifiers 2204 where the eyes can be magnified to assist theuser/examiner. These synchronized magnifiers can be moved to any part ofthe faces and are correlated to show the same portions of the faces 104and 204 at the same time. FIG. 23 shows an exemplary embodiment of afirst increased zoom feature 2304 of the embedded synchronizedmagnifiers and FIG. 24 shows a second exemplary embodiment of anincreased zoom 2404 of embedded synchronized magnifiers. Here, theexaminer has zoomed in significantly on one each in each of the probeand candidate images.

As each facial component is displayed to the user, a pair ofsynchronized magnification areas can optionally be displayed over theimages allowing the user to zoom in on any discrete area, as shown, forexample, in the above figures. At any time the size and zoom of themagnifiers can be controlled, updated or modified by the user.

Step 5:

In one embodiment, the user cannot complete the facial identificationprocess unless:

-   -   All facial components in the list have been reviewed by the        user; and    -   The user has specified whether for each facial component there        is a similarity, a difference, or it cannot be evaluated.

The user can Exit the facial identification process at any stage in theprocess, but a decision as to whether the same person is in both imagescannot be evaluated in such an event.

At the end of the entire facial identification process, a permanentauditing record may optionally be saved which confirms:

-   -   Two images were reviewed in accordance with a predefined process        (e.g., guidelines or best practices);    -   On a specific date, a user on an operational workstation        performed the facial identification controlled by the predefined        process; and    -   A final decision by the human examiner was arrived at.

FIG. 25 illustrates an exemplary system that can be used in conjunctionwith the above steps. The exemplary system includes aprocessor/controller 2504, memory 2508, I/O interface 2512, GPU 2516,facial components database 2520, facial component manager 2524, facialcomponent list manager, verifier and auditor 2528, magnifier module2532, folder manager 2536, dynamic report generator 2540, audit reportmodule 2544, guidelines 2548, one or more displays 2552, one or more I/Odevices 2556, and one or more remote biometricsystem(s)/databases/back-end systems 2560 (connected via thenetwork(s)).

In operation, and in accordance with one exemplary embodiment, thesystem can be configured to resemble an email system. While otherconfigurations are certainly possible, the emulation of an email systemin terms of workflow at least assists with management and organizationof the various images.

In accordance with one exemplary embodiment, the facial componentmanager 2524, in conjunction with the processor 2504, memory 2508, GPU2516 and facial components database 2520 present a probe sample to theuser at display 2552. This probe sample can be forwarded to a back-endsystem, such as remote biometric system 2560 to perform initialmatching. Once sent, the folder manager 2536 can place the sent probesample in a “sent folder”—the “sent folder” representing that the probesample has been sent away for a search to be done by the back-end system2560. In conjunction with the search being sent out, a demographicfilter and/or other filtering criteria can be sent with the image tohelp reduce the number of false positives received back from theback-end system.

On receipt of the results from the back-end system 2560, an indicationcan be placed in an “Inbox” by the folder manager 2536 alerting theuser/examiner that there are probe and candidate samples ready to review(these results can be stored in the facial components database 2520).The GPU 2516 can provide to the user/examiner, on displays 2552, acandidate screen that shows all candidates returned by the back-endsystem 2560. This candidate screen, in conjunction with the facialcomponents database 2520 and facial component manager 2524, can befiltered to reduce the number of candidates. For example, auser/examiner can select Yes, No or Maybe, to reduce the number ofcandidates to a more manageable number with the No candidates optionallybeing removed from the display.

Next, the user/examiner can perform a detailed review of the probesample to one or more of the candidates as shown in FIGS. 3-25. Duringthe comparison, the user/examiner can one or more of calibrate thefaces, perform a visual alignment of the faces, adjust scaling, adjustcolors, and in general adjust any one or more of the characteristics ofthe probe and candidate sample either in unison or separately. Ingeneral any type of image adjustment can be performed including one ormore of invert, color channel adjustment, greyscale adjustment, CYMKadjustment, or the like.

As the user/examiner reviews each of the facial characteristics, thefacial component list manager, verifier and auditor 2528, in conjunctionwith the processor 2504, facial component manager 2524 and facialcomponent database 2520, tracks the actions and records the results ofthe comparisons. The user/examiner during this process can takesnapshots of particular features of interest with these snapshots beingcaptured and added into a report which is being dynamically generated bythe dynamic report generator 2540. These snapshots can then optionallybe supplemented with one or more comments from the user/examiner.

Also, at any time, the guidelines 2548 can be retrieved and displayed tothe user/examiner. These guidelines can be dynamically liked to facialfeatures such that if the user/examiner clicks on a particular feature(such as the eyes) the corresponding guidelines (such as in a PDF, textor other document format) for eye comparison can be displayed—such asthe guideline shown in FIGS. 10-21 in windows 404.

Optionally, the audit report module 2544 can keep track and store one ormore actions taken by the user/examiner, with the stored informationusable for one or more of training, to confirm lack of biases, toidentify errors and/or to ensure any applicable guidelines have beencomplied with. This audit trail can then be stored in the facialcomponents database 2520 (or in general at any location and/or printedor sent to a destination such as via email) along with any informationabout the probe and/or candidate.

Optionally, the dynamic report generator 2540 can generate one or morereports. These reports can be compiled at the end of the review process(e.g., a finalization report) and/or dynamically created as thecomparisons are made. As one example shows in FIG. 26, the report 2604includes one or more sections populated with information regardingcomparison decisions that were made. The exemplary sections include:“Snapshot”, “Feature”, “Same/Different”, “Snapshot Images”, “Comments”,and “Data Path.”. Snapshot can indicate a sequence number for thesnapshot. Feature can indicate which feature the snapshot is for.Same/Different can represent whether the examiner believes the featureis the same or different. Snapshot images can include images of thecomparison. Comments can include any examiner comments and data path caninclude information regarding the source file for the captured images.

As discussed above, the system can optionally be configured in anemail-like manner. Thus, the system can have, for example, a reviewedfolder, indicating that the examiner has reviewed and/or completed thecomparison. The system can also optionally include a “needs attention”folder for situations where there has been an anomalous error.

For all of the various information, captures, reports etc. describedherein, metadata can be associated with that information and canoptionally be used for sorting (name, date, ID, etc.) or the like.

FIG. 27 outlines an exemplary method for performing facial recognition.In particular, control begins in step S2704 and continues to step S2708.In step S2708 a probe sample is loaded for display. Next, in step S2712,one or more candidate samples are loaded for display. Then, in StepS2716, the samples can be displayed in a side-by-side or other manner.Control then continues to step S2720.

In step S2720, a predefined list of facial components to be compared ismaintained and managed. As discussed, an exemplary embodiment caninclude one or more of the following facial components that requirecomparisons: skin, face/head outline, face/head composition, hairline orbaldness pattern, forehead, eyebrows, eyes, cheeks, nose, ears, mouth,chin or jaw line, neck, facial hair, facial lines, scars, facial marks,and alterations. Next, in step S2724, the user is guided through theexamination process with reference to the predefined list. Then, in stepS2728, the user is optionally provided one or more ofguidelines/reference material/instructions/examples for the specificcomparison being made. Optionally further, the user can have theirattention drawn to a specific facial feature being compared. Optionallystill, the system can automatically update the display with one or moreof the guidelines/reference material/instructions/examples/drawings tocorrespond to the facial component currently being examined. Controlthen continues to step S2732.

In step S2732, the predefined list can be updated as each facialcomponent comparison is completed. Next, in step S2736, the user canoptionally be required to complete a current comparison before beingallowed to proceed to a next comparison. Then, in step S2740, useractions can optionally be logged as discussed. Control then continues tostep S2744.

In step S2744, the user can optionally record one or more ofnotes/comments/images/comparison results associated with one or more ofthe comparisons—with one or more of these items optionally being addedto a report. Next, in step S2748, the user can optionally be required toperform all comparisons for completion of the facial identificationprocess—with the user optionally being returned to complete anyincomplete comparisons. Then, in step S2752 an audit report canoptionally be generated. Control then continues to step S2756. In stepS2756, a finalization report can be generated with control continuing tostep S2760 where the control sequence ends.

In accordance with yet another optional embodiment, the system caninclude an application that can run on a desktop (e.g., ForensicSpectacles) which allows a user to control and directly push any twoimages currently displayed anywhere on the desktop directly into a 1:1examination screen. This allows a user to, for example, see facialexpressions—this allows “one click facial review.” In another exemplaryembodiment, the linked magnifiers can “snap” examination areas ofinterest into an off screen repository where they can then be directlypublished into a finalization document. This allows users to create thefinal examination report as part of their workflow which can, forexample, dramatically shorten the time to create the final report.

In accordance with another exemplary embodiment, the checklist(predefined list) can also lay out the facial components. Another windowcan optionally be shown that references the correct and acceptedanatomical definitions of the facial components. Both the checklist andthe definitions can be synced to the facial component the examiner iscurrently reviewing. This can provide for each facial component 1) achecklist of what to look for, and 2) the proper anatomical terms forthe specific component. Therefore, the examiner can be given a fullsuite of items and proper anatomical definitions for managing the facialidentification. FIGS. 28-37 show examples of accepted anatomicaldefinitions of the facial components which are available fromfiswg.org—Facial Image Comparison Feature List for MorphologicalAnalysis, which is incorporated herein by reference in its entirety.

Additionally, the following two documents, which are also incorporatedherein by reference in their entirety, provide information andbackground: SWGFAST, Document #10, Standards for Examining FrictionRidge, Impressions and Resulting Conclusions (Latent/Tenprint) atwww.swgfast.org/documents/examinations-conclusions/121124_Examinations-Conclusions_2.0.pdfand SWGFAST, Document #8, Standard for the Documentation of Analysis,Comparison, Evaluation, and Verification (ACE-V) atwww.swgfast.org/documents/documentation/121124_Standard-Documentation-ACE-V_2.0.pdf.

Exemplary aspects are directed toward:

A method of performing a facial identification process, the methodcomprising:

displaying two facial images on a display screen adjacent to each other;

electronically maintaining a predefined list of facial components to beexamined when the two facial images on the display screen are analyzedby a user to determine whether the facial images are of differentpersons or of a same person; and

automatically guiding the user through the facial identification processin response to the electronically maintained list of facial components.

Any of the above aspects, wherein the list of facial components includesskin, face/head outline, face/head composition, hairline or baldnesspattern, forehead, eyebrows, eyes, cheeks, nose, ears, mouth, chin orjaw line, neck, facial hair, facial lines, scars, facial marks, andalterations.

Any of the above aspects, further comprising requiring the user tocomplete examination of a facial component currently being examinedbefore examination can proceed to another facial component in the listof facial components.

Any of the above aspects, further comprising synchronizing the displayof the two facial images with a facial component currently beingexamined by the user.

Any of the above aspects, wherein the synchronizing the display of thetwo facial images with the facial component currently being examined bythe user includes electronically drawing attention to the current facialcomponent in both of the facial images.

Any of the above aspects, wherein electronically drawing attention tothe current facial component in both of the facial images includes oneor more of: drawing a box around the current facial component, pointingan arrow to the current facial component, correlating the facialcomponent to pertinent reference material, magnifying the facialcomponent being reviewed, placing the two facial components in a waywhich is easier for the user to compare, blurring the facial areas notbeing reviewed, and highlighting the current facial component withcolor.

Any of the above aspects, further comprising displaying the list offacial components on a display screen, while the two facial images arebeing displayed.

Any of the above aspects, further comprising electronically displayingone or more of a description, instructions, and drawings related toperforming an examination of a facial component currently being examinedby the user, while the two facial images are being displayed.

Any of the above aspects, further comprising automatically updating thedisplay of the one or more of a description, instructions, and drawingsto correspond to the facial component currently being examined.

Any of the above aspects, further comprising generating an audit recordcontaining every facial component in the list that was examined by theuser.

Any of the above aspects, further comprising requiring the user toexamine every facial component in the list of facial components in orderto complete the facial identification process.

Any of the above aspects, further comprising receiving input from theuser that centers eye locations in each of the displayed facial images.

Any of the above aspects, further comprising maintaining a viewed statusand a comparison results status for each of the facial components in thelist.

Any of the above aspects, wherein the automatically guiding the userthrough the facial identification process includes examining the facialcomponents in the predefined list of facial components sequentially.

Any of the above aspects, further comprising magnifying a facialcomponent currently being examined in both of the displayed facialimages.

Computer program product for performing a facial identification process,the computer program product comprising:

a non-transitory computer readable storage medium having computerreadable program code embodied therewith, the computer readable programcode comprising:

computer readable program code that, if executed, displays two facialimages on a display screen adjacent to each other;

computer readable program code that, if executed, provides a predefinedlist of facial components to be examined when the two facial images onthe display screen are analyzed by a user to determine whether thefacial images are of different persons or of a same person;

computer readable program code that, if executed, automatically guidesthe user through the facial identification process based on the list offacial components.

Any of the above aspects, wherein the list of facial components includesskin, face/head outline, face/head composition, hairline or baldnesspattern, forehead, eyebrows, eyes, cheeks, nose, ears, mouth, chin orjaw line, neck, facial hair, facial lines, scars, facial marks, andalterations.

Any of the above aspects, further comprising computer readable programcode that, if executed, requires the user to complete examination of afacial component currently being examined before examination can proceedto another facial component in the list of facial components.

Any of the above aspects, further comprising computer readable programcode that, if executed, synchronizes the display of the two facialimages with a facial component currently being examined by the user.

Any of the above aspects, wherein the computer readable program codethat, if executed, synchronizes the display of the two facial imageswith the facial component currently being examined by the user includescomputer readable program code that, if executed, electronically drawsattention to the current facial component in both of the facial images.

Any of the above aspects, wherein the computer readable program codethat, if executed, electronically draws attention to the current facialcomponent in both of the facial images includes computer readableprogram code to perform one or more of: drawing a box around the currentfacial component, pointing an arrow to the current facial component,correlating the facial component to pertinent reference material,magnifying the facial component being reviewed, placing the two facialcomponents in a way which is easier for the user to compare, blurringthe facial areas not being reviewed, and highlighting the current facialcomponent with color.

Any of the above aspects, further comprising computer readable programcode that, if executed, displays the list of facial components on adisplay screen, while the two facial images are being displayed.

Any of the above aspects, further comprising computer readable programcode that, if executed, electronically displays one or more of adescription, instructions, and drawings related to performing anexamination of a facial component currently being examined by the user,while the two facial images are being displayed.

Any of the above aspects, further comprising computer readable programcode that, if executed, automatically updates the display of the one ormore of a description, instructions, and drawings to correspond to thefacial component currently being examined.

Any of the above aspects, further comprising computer readable programcode that, if executed, generates an audit record containing everyfacial component in the list that was examined by the user.

Any of the above aspects, further comprising computer readable programcode that, if executed, requires the user to examine every facialcomponent in the list of facial components in order to complete thefacial identification process.

Any of the above aspects, further comprising computer readable programcode that, if executed, receives input from the user that centers eyelocations in each of the displayed facial images.

Any of the above aspects, further comprising computer readable programcode that, if executed, maintains a viewed status and a comparisonresults status for each of the facial components in the list.

Any of the above aspects, wherein the computer readable program codethat, if executed, automatically guides the user through the facialidentification process includes computer readable program code that, ifexecuted, sequentially examines the facial components in the predefinedlist of facial components.

Any of the above aspects, further comprising computer readable programcode that, if executed, magnifies a facial component currently beingexamined in both of the displayed facial images.

A computer system of performing a facial identification process, thecomputer system comprising:

memory storing program code used to perform a facial identificationprocess;

a processor executing the program code stored in the memory to:

display two facial images on a display screen adjacent to each other;

electronically maintain a predefined list of facial components to beexamined when the two facial images on the display screen are analyzedby a user to determine whether the facial images are of differentpersons or of a same person; and

automatically guide the user through the facial identification processin response to the electronically maintained list of facial components.

Any of the above aspects, wherein the computer system is configured toperform any of the above aspects.

A facial identification system to automatically assist with performing afacial identification process comprising:

an I/O interface in communication with one or more biometric systems anddatabases, the I/O interface receiving image information includingfacial images;

at least one display and graphics processing unit in communication thatcooperate to display at least two facial images near one other;

a processor and storage in electrical communication that electronicallymaintain a predefined list of facial components to be examined when thetwo facial images on the at least one display are analyzed by a user todetermine whether the facial images are of different persons or of asame person;

a guideline database; and

a facial component list manager that automatically guides the userthrough the facial identification process in response to theelectronically maintained list of facial components, wherein theautomatic guiding includes providing one or more of instructions,guidelines and anatomical definitions from the guideline database to theuser and keeping track of which facial components have been examined.

Any of the above aspects, wherein the facial images are managed in oneor more folders including a sent folder, an inbox folder and a reviewedfolder.

A system for performing a facial identification process comprising:

means for displaying two facial images on a display screen adjacent toeach other;

means for electronically maintaining a predefined list of facialcomponents to be examined when the two facial images on the displayscreen are analyzed by a user to determine whether the facial images areof different persons or of a same person; and

means for automatically guiding the user through the facialidentification process in response to the electronically maintained listof facial components.

One or more means for performing any one or more of the above functions.

Any one or more of the features as substantially described herein.

For purposes of explanation, numerous details are set forth in order toprovide a thorough understanding of the present embodiments. It shouldbe appreciated however that the techniques herein may be practiced in avariety of ways beyond the specific details set forth herein.

Furthermore, while the exemplary embodiments illustrated herein may showthe various components of the system collocated, it is to be appreciatedthat the various components of the system can be located at distantportions of a distributed network, such as a communications networkand/or the Internet, or within a dedicated secure, unsecured and/orencrypted system. Thus, it should be appreciated that the components ofthe system can be combined into one or more devices, or collocated on aparticular node/element(s) of a distributed network, such as acommunications network. As will be appreciated from the description, andfor reasons of computational efficiency, the components of the systemcan be arranged at any location within a distributed network withoutaffecting the operation of the system.

Furthermore, it should be appreciated that the various links, includingcommunications channel(s), connecting the elements (which may not be notshown) can be wired or wireless links, or any combination thereof, orany other known or later developed element(s) that is capable ofsupplying and/or communicating data and/or signals to and from theconnected elements. The term module as used herein can refer to anyknown or later developed hardware, software, firmware, or combinationthereof that is capable of performing the functionality associated withthat element. The terms determine, calculate and compute, and variationsthereof, as used herein are used interchangeably and include any type ofmethodology, process, mathematical operation or technique.

While the above-described flowcharts/operational flows have beendiscussed in relation to a particular exemplary sequence of events, itshould be appreciated that changes to this sequence can occur withoutmaterially effecting the operation of the embodiment(s). Additionally,the exact sequence of events need not occur as set forth in theexemplary embodiments, but rather the steps can be performed by one orthe other device(s) in the system. Additionally, the exemplarytechniques illustrated herein are not limited to the specificallyillustrated embodiments but can also be utilized with the otherexemplary embodiments and each described feature is individually andseparately claimable.

As will be appreciated by one skilled in the art, aspects of the presentdisclosure may be embodied as a system, method, and/or computer programproduct. Thus, aspects of the present disclosure may be embodiedentirely in hardware, entirely in software (including, but not limitedto, firmware, program code, resident software, microcode), or in acombination of hardware and software. All such embodiments may generallybe referred to herein as a circuit, a module, or a system. In addition,aspects of the present invention may be in the form of a computerprogram product embodied in one or more computer readable media havingcomputer readable program code embodied thereon.

The computer readable medium may be a computer readable storage medium,examples of which include, but are not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination thereof. As usedherein, a computer readable storage medium may be any non-transitory,tangible medium that can contain or store a program for use by or inconnection with an instruction execution system, apparatus, device,computer, computing system, computer system, or any programmable machineor device that inputs, processes, and outputs instructions, commands, ordata. A non-exhaustive list of specific examples of a computer readablestorage medium include an electrical connection having one or morewires, a portable computer diskette, a floppy disk, a hard disk, arandom access memory (RAM), a read-only memory (ROM), a USB flash drive,an non-volatile RAM (NVRAM or NOVRAM), an erasable programmableread-only memory (EPROM or Flash memory), a flash memory card, anelectrically erasable programmable read-only memory (EEPROM), an opticalfiber, a portable compact disc read-only memory (CD-ROM), a DVD-ROM, anoptical storage device, a magnetic storage device, or any suitablecombination thereof. A computer readable storage medium can be anycomputer readable medium that is not a computer readable signal mediumsuch as a propagated data signal with computer readable program codeembodied therein.

Program code may be embodied as computer-readable instructions stored onor in a computer readable storage medium as, for example, source code,object code, interpretive code, executable code, or combinationsthereof. Any standard or proprietary, programming or interpretivelanguage can be used to produce the computer-executable instructions.Examples of such languages include C, C++, C#, Pascal, JAVA, JAVAScript, BASIC, Smalltalk, Visual Basic, and Visual C++.

Transmission of program code embodied on a computer readable medium canoccur using any appropriate medium including, but not limited to,wireless, wired, optical fiber cable, radio frequency (RF), or anysuitable combination thereof.

The program code may execute entirely on a user's computer, partly onthe user's computer, as a stand-alone software package, partly on theuser's computer and partly on a remote computer or entirely on a remotecomputer or server. Any such remote computer may be connected to theuser's computer through any type of network, including a local areanetwork (LAN) or a wide area network (WAN), or the connection may bemade to an external computer (for example, through the Internet using anInternet Service Provider).

Additionally, the systems, methods and protocols can be implemented toimprove one or more of a special purpose computer, a programmedmicroprocessor or microcontroller and peripheral integrated circuitelement(s), an ASIC or other integrated circuit, a digital signalprocessor, a hard-wired electronic or logic circuit such as discreteelement circuit, a programmable logic device such as PLD, PLA, FPGA,PAL, any comparable means, or the like. In general, any device capableof implementing a state machine that is in turn capable of implementingthe methodology illustrated herein can benefit from the variouscommunication methods, protocols and techniques according to thedisclosure provided herein.

Examples of the processors as described herein may include, but are notlimited to, at least one of Qualcomm® Snapdragon® 800 and 801, Qualcomm®Snapdragon® 610 and 615 with 4G LTE Integration and 64-bit computing,Apple® A7 processor with 64-bit architecture, Apple® M7 motioncoprocessors, Samsung® Exynos® series, the Intel® Core™ family ofprocessors, the Intel® Xeon® family of processors, the Intel® Atom™family of processors, the Intel Itanium® family of processors, Intel®Core® i5-4670K and i7-4770K 22 nm Haswell, Intel® Core® i5-3570K 22 nmIvy Bridge, the AMD® FX™ family of processors, AMD® FX-4300, FX-6300,and FX-8350 32 nm Vishera, AMD® Kaveri processors, Texas Instruments®Jacinto C6000™ automotive infotainment processors, Texas Instruments®OMAP™ automotive-grade mobile processors, ARM® Cortex™-M processors,ARM® Cortex-A and ARIV1926EJS™ processors, Broadcom® AirForceBCM4704/BCM4703 wireless networking processors, the AR7100 WirelessNetwork Processing Unit, other industry-equivalent processors, and mayperform computational functions using any known or future-developedstandard, instruction set, libraries, and/or architecture.

Furthermore, the disclosed methods may be readily implemented insoftware using object or object-oriented software developmentenvironments that provide portable source code that can be used on avariety of computer, workstation or mobile device platforms.Alternatively, the disclosed system may be implemented partially inhardware using standard logic circuits or a VLSI design. Whethersoftware or hardware is used to implement the systems in accordance withthis invention is dependent on the speed and/or efficiency requirementsof the system, the particular function, and the particular software orhardware systems or microprocessor or microcomputer systems beingutilized. The methods illustrated herein however can be readilyimplemented in hardware and/or software using any known or laterdeveloped systems or structures, devices and/or software by those ofordinary skill in the applicable art from the functional descriptionprovided herein and with a general basic knowledge of the computer andimage processing arts.

Moreover, the disclosed methods may be readily implemented in softwareexecuted on programmed general-purpose computer, a special purposecomputer, mobile device, smartphone, a microprocessor, or the like. Inthese instances, the systems and methods of this invention can beimplemented as program embedded on personal computer such as JAVA® orCGI script, as a resource residing on a server or graphics workstation,as a routine embedded in a dedicated fingerprint processing system, as aplug-in, or the like. The system can also be implemented by physicallyincorporating the system and method into a software and/or hardwaresystem, such as the hardware and software systems of an image processor.

While this invention has been described in conjunction with a number ofembodiments, it is evident that many alternatives, modifications andvariations would be or are apparent to those of ordinary skill in theapplicable arts. Accordingly, it is intended to embrace all suchalternatives, modifications, equivalents, and variations that are withinthe spirit and scope of this invention.

The invention claimed is:
 1. A method of providing electronicallyassisted facial identification comprising: displaying, on at least onedisplay for a human facial examiner, at least two facial images;electronically maintaining, by a processor and storage, a predefinedlist of facial components to be examined by the human facial examiner inthe least two facial images, the predefined list of facial componentsguiding the human facial examiner to assist the human facial examinerwith determining whether the facial images are of different persons orof a same person; and automatically guiding the human facial examinerthrough the facial identification process in response to interactionwith the electronically maintained list of facial components, whereinthe automatic guiding includes providing one or more of instructions,guidelines and anatomical definitions to the human facial examiner, andmaintaining in storage an electronic database of which facial componentshave been examined.
 2. The method of claim 1, wherein the list of facialcomponents includes one or more of: skin, face/head outline, face/headcomposition, hairline or baldness pattern, forehead, eyebrows, eyes,cheeks, nose, ears, mouth, chin or jaw line, neck, facial hair, faciallines, scars, facial marks, and alterations.
 3. The method of claim 1,further comprising requiring the human facial examiner to completeexamination of a facial component currently being examined beforeexamination can proceed to another facial component in the list offacial components.
 4. The method of claim 1, further comprisingautomatically synchronizing the display of the two facial images with afacial component currently being examined by the human facial examiner.5. The method of claim 4, wherein the synchronizing the display of thetwo facial images with the facial component currently being examined bythe human facial examiner includes automatically electronically drawingattention to the current facial component in both of the facial images.6. The method of claim 5, wherein electronically drawing attention tothe current facial component in both of the facial images includes oneor more of: drawing a box around the current facial component, pointingan arrow to the current facial component, correlating the facialcomponent to pertinent reference material, magnifying the facialcomponent being reviewed, placing the two facial components in a waywhich is easier for the human facial examiner to compare the components,blurring the facial areas not being reviewed, and highlighting thecurrent facial component with color.
 7. The method of claim 1, furthercomprising displaying the list of facial components on the displayscreen, while the two facial images are being displayed.
 8. The methodof claim 1, further comprising automatically and dynamicallyelectronically displaying one or more of a description, theinstructions, and drawings related to performing an examination of afacial component currently being examined by the human facial examiner,while the two facial images are being displayed.
 9. The method of claim8, further comprising automatically updating the display of the one ormore of a description, instructions, and drawings to correspond to thefacial component currently being examined.
 10. The method of claim 1,further comprising generating an audit record containing every facialcomponent in the list that was examined by the human facial examiner.11. The method of claim 1, further comprising requiring the human facialexaminer to examine every facial component in the list of facialcomponents in order to complete the facial identification process. 12.The method of claim 1, further comprising receiving input from the humanfacial examiner that centers eye locations in each of the displayedfacial images.
 13. The method of claim 1, further comprising maintaininga viewed status and a comparison results status for each of the facialcomponents in the list.
 14. The method of claim 1, wherein theautomatically guiding the human facial examiner through the facialidentification process includes examining the facial components in thepredefined list of facial components sequentially.
 15. The method ofclaim 1, further comprising magnifying a facial component currentlybeing examined in both of the displayed facial images.
 16. A facialidentification system to automatically assist with performing a facialidentification process comprising: at least one display and graphicsprocessing unit that cooperate to display at least two facial imagesnear one other; a processor and storage in electrical communication thatelectronically maintain a predefined list of facial components to beexamined when the two facial images on the at least one display areanalyzed by a human facial examiner to determine whether the facialimages are of different persons or of a same person; and a facialcomponent list manager that automatically guides the human facialexaminer through the facial identification process in response to theelectronically maintained list of facial components, wherein theautomatic guiding includes providing one or more of instructions,guidelines and anatomical definitions to the human facial examiner andkeeping track of which facial components have been examined by the humanfacial examiner.
 17. The system of claim 16, wherein the list of facialcomponents includes one or more of: skin, face/head outline, face/headcomposition, hairline or baldness pattern, forehead, eyebrows, eyes,cheeks, nose, ears, mouth, chin or jaw line, neck, facial hair, faciallines, scars, facial marks, and alterations.
 18. The system of claim 16,further comprising a facial component auditor that requires the humanfacial examiner to complete examination of a facial component currentlybeing examined before examination can proceed to another facialcomponent in the list of facial components.
 19. The system of claim 16,wherein the graphics processing unit automatically synchronizes thedisplay of the two facial images with a facial component currently beingexamined by the human facial examiner.
 20. The system of claim 19,wherein the synchronizing the display of the two facial images with thefacial component currently being examined by the human facial examinerincludes automatically electronically drawing attention to the currentfacial component in both of the facial images.
 21. The system of claim20, wherein electronically drawing attention to the current facialcomponent in both of the facial images includes one or more of: drawinga box around the current facial component, pointing an arrow to thecurrent facial component, correlating the facial component to pertinentreference material, magnifying the facial component being reviewed,placing the two facial components in a way which is easier for the humanfacial examiner to compare, blurring the facial areas not beingreviewed, and highlighting the current facial component with color. 22.The system of claim 16, wherein the facial component list managerdisplays the list of facial components on a display screen, while thetwo facial images are being displayed.
 23. The system of claim 16,further comprising automatically and dynamically electronicallydisplaying one or more of a description, the instructions, and drawingsrelated to performing an examination of a facial component currentlybeing examined by the human facial examiner, while the two facial imagesare being displayed.
 24. The system of claim 23, further comprising afacial component verifier that automatically updates the display of theone or more of a description, instructions, and drawings to correspondto the facial component currently being examined.
 25. The system ofclaim 16, further comprising an audit report module in electricalcommunication with the processor to generate an audit record containingevery facial component in the list that was examined by the human facialexaminer.
 26. The system of claim 16, further comprising a facialcomponent auditor that requires the human facial examiner to examineevery facial component in the list of facial components in order tocomplete the facial identification process.
 27. The system of claim 16,further comprising an I/O device that receives input from the humanfacial examiner that centers eye locations in each of the displayedfacial images.
 28. The system of claim 16, further comprising a facialcomponent auditor that maintains in the storage a viewed status and acomparison results status for each of the facial components in the list.29. The system of claim 16, wherein the automatically guiding the humanfacial examiner through the facial identification process includesexamining the facial components in the predefined list of facialcomponents sequentially.
 30. The system of claim 16, further comprisinga magnifier module that magnifies a facial component currently beingexamined in both of the displayed facial images.